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REMARKS 

INTRODUCTION 

Claims 1-26 were previously pending and under consideration. 
Claims 27-32 are added herein. 

Therefore, claims 1-32 are now pending and under consideration. 
Claims 1-22 are rejected. 

Claims 1, 8-10, 12, 19-21 and 23 are amended herein. 

No new matter is being presented, and approval and entry are respectfully requested. 

ENTRY OF AMENDMENT UNDER 37 CFR §1.116 

Applicant requests entry of this Rule 116 Response because: 

(a) the rejected claims have been canceled; 

(b) it is believed that the amendment of the claims puts this application into condition 
for allowance as suggested by the Examiner; 

(c) the amendments were not earlier presented because the Applicant believed in 
good faith that the cited prior art did not disclose the present invention as previously claimed; 

(d) the amendments of the claims should not entail any further search by the 
Examiner since no new features are being added or no new issues are being raised; and 

(e) the amendments do not significantly alter the scope of the claims and place the 
application at least into a better form for purposes of appeal. No new features or new issues 
are being raised. 

The Manual of Patent Examining Procedures sets forth in Section 714.12 that "any 
amendment that would place the case either in condition for allowance or in better form for 
appeal may be entered." Moreover, Section 714.13 sets forth that "the Proposed 
Amendment should be given sufficient consideration to determine whether the claims are in 
condition for allowance and/or whether the issues on appeal are simplified." The Manual of 
Patent Examining Procedures further articulates that the reason for any non-entry should be 
explained expressly in the Advisory Action. 
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TRAVERSAL OF RESTRICTION 

Claims 23-26 were restricted and constructively elected as being directed to a 
separate invention. The restriction is traversed and withdrawal of the restriction and 
consideration of the claims is respectfully requested. 

The restriction is traversed because group II is not properly placed in class 382, 
subclass 154, but rather belongs with group I in subclass 153. The Manual of Patent 
Classification describes subclass 154 as "[s]ubject matter wherein a three-dimensional scene 
is imaged using at least two cameras or camera locations for the generation of XYZ 
coordinate data of any object within the scene ". In sum, subclass 154 is for virtual scene 
generation. Nothing in any of claims 23-26 suggests that XYZ coordinate data of any object 
in a scene can be determined. Claim 23, for example, recites determining an arrangement of 
an object relative to a robot. Group II properly belongs in the same class as group I; 
subclass 153. Subclass 153 is directed to "[s]ubject matter related to ... positioning articles 
by automated manufacturing systems using image analysis". Claim 23 recites "[a] method for 
automatically determining an arrangement of a workpiece relative to a robot", which is very 
similar to claim Vs "robot system having an image processing function for determining 
orientation, or orientation and position of a robot operation on an object of detection among a 
plurality of objects". Both claims 1 and 23 recite image matching, which is a well-known type 
of image analysis. Applicant respectfully requests an explanation of what features would 
enable generation of coordinates of any object in an imaged scene, and how claim 23 differs 
from claim 1 such that it belongs in a subclass other than subclass 153. 

The restriction is also traversed because the cited particulars of group II are either 
found in group I or have been deleted. The restriction noted that group I does not have an 
image device affixed to a robot arm. This feature has been cancelled from group II. The 
restriction proposed that group I does not "[store] images of an [sic] workpiece/object shaped 
like the workpiece in the reference image". However, claim 1 recites "storing reference 
models, each created based on an image of a reference object ... said reference object 
being the object of detection or having a shape substantially identical to that of the object of 
detection". The restriction also proposed that group I does not "[capture] a working image of 
the workpiece from a known arrangement of the robot". However, claim 1 recites "an image 
of the object of detection (captured by said first image capturing device)", and claim 7 recites 
"said robot situates said second image data capturing device to have said determined 
orientation [e.g. a known orientation] or to have said determined orientation and said 
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determined position with respect to the object, so that said second image data capturing 
device is directed to a characterizing portion of the object; and wherein said processor 
detects three-dimensional position and/or posture [e.g. arrangement] of the object based on 
three-dimensional position of said characterizing portion obtained by said second image 
capturing device". 

The restriction also proposed that the subcombination has a separate utility of 
determining an arrangement of a person or other specific object or workpiece relative to an 
image capturing device. First, group II does not recite determining an arrangement relative 
to an image capturing device. Second, claim 1 recites "determining] the orientation, or the 
orientation and position of the robot operation to be performed on the object of detection". 
Applicant respectfully requests how this utility of claim 1 differs from claim 23's "determining 
an arrangement of a workpiece relative to a robot" 

Entry and consideration of claims 23-26 is respectfully requested. 

Applicant respectfully notes that "[w]here the initial requirement is traversed, it should 
be reconsidered. If, upon reconsideration, the examiner is still of the opinion that restriction is 
proper, it should be repeated and made final in the next Office action. (See MPEP § 803.01.) 
In doing so, the examiner should reply to the reasons or arguments advanced by applicant in 
the traverse." MPEP § 821 .01 . Applicant respectfully requests a separate reply to each of 
the arguments presented above. 

OBJECTIONS TO THE DRAWINGS 

In the Office Action, at page 3, the proposed drawing correction was objected to. In 
view of the accompanying separate Attachment to the Amendment, corrections to Figures 6 
and 9 have been requested. Therefore, the outstanding drawing objections should be 
resolved. 

Reconsideration and withdrawal of the outstanding objections to the drawings are 
respectfully requested. 
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REJECTIONS UNDER 35 USC § 112, SECOND PARAGRAPH 

In the Office Action, at page 6, claims 8-10 and 19-21 were rejected under 35 U.S.C. 
§ 1 12, second paragraph, for the reasons set forth therein. Claims 8-10/19-21 are amended 
from their true original form in view of the Preliminary Amendment filed April 6, 2003 (i.e. 
dependence from claim 1/12) to now depend from claim 6/17. Withdrawal of the rejection is 
respectfully requested. 

NEW DEPENDENT CLAIMS 27-32 

For the convenience of the Examiner, it is noted that new dependent claims 27-29 are 
similar to claims 8-10, and new dependent claims 30-32 are similar to claims 1 9-21 . 

/* 

REJECTIONS UNDER 35 USC § 103 

In the Office Action, at pages 6-9, claims 1-4 and 12-15 were rejected under 35 
U.S.C. § 103 as obvious over Suzuki in view of Kelley. This rejection is traversed and 
reconsideration is requested. 

PRIOR ART DOES NOT MATCH IMAGE OF OBJECT OF DETECTION WITH 
PREDETERMINED REFERENCE MODELS BASED ON IMAGE OF REFERENCE OBJECT 
CAPTURED IN PLURALITY OF DIFFERENT DIRECTIONS TO DETERMINE 
ORIENTATION OF THE OBJECT OF DETECTION 

Claim 1 , for example, recites "determining orientation ... of a robot operation on an 
object of detection among a plurality of objects", by "storing reference models, each 
comprising an image of a reference object captured by said image capturing device in a 
different direction [and information of the same]", and by performing "matching processing on 
the reference model images and a working image of the plurality of objects (captured by said 
first image capturing device) to select an image of one of the reference models that matches 
the object of detection in the working image" whose reference model (having capture 
direction information) is then used to determine "the orientation of the robot operation to be 
performed on the object of detection". Claim 12 recites similar features with plural kinds of 
objects. See also claim 23. 

The rejection cites Kelley as adding to Suzuki the matching of an image of a 
workpiece captured during operation to a reference model created during instruction. The 
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rejection cites Suzuki for the robotic acquisition of model images from different directions, 
and cites Kelley for matching reference models to an image of the workpiece to determine 
the workpiece's orientation relative to the robot. However, neither reference discusses or 
suggests using predetermined workpiece-robot arrangement information (reference models) 
to orient the robot for the purpose of acquiring or preparing to acquire the workpiece. In 
Kelley, the workpiece is picked up without knowing its orientation. In Suzuki, there is no 
indication that the orientation of the workpiece is known when it is picked up, or if it is known, 
how it is determined. 

Neither Suzuki nor Kelley use predetermined workpiece-robot position/orientation 
information to orient the robot for picking up or preparing to pick up the workpiece. In Suzuki, 
there is little discussion of how the robot acquires a part on the conveyor belt. The position is 
determined by a CCD, but the orientation of the robot is not discussed. In Kelley, the part- 
robot relative arrangement is determined after the part has been picked up. 

Suzuki discusses three-dimensional positions of the vertices and edges of the sample 
product 10, which is made up of a plurality of parts. The edge and vertex information is 
based on analysis of two-dimensional images obtained by imaging the sample product 10 
from various directions by a pair of CCD cameras. The captured images are not used after 
deconstruction. Suzuki does not discuss or suggest performing matching process with an 
image of the object of detection using reference models that are predetermined based on 
images of a reference object captured in a plurality of different directions (thereby 
determining the orientation of the object of detection). Suzuki discusses capturing images 
from different directions to derive the construction of the parts. The robot orientations during 
capture are not used during operation of the robot. Construction operations are 
predetermined based on the derived construction. The parts are then sequentially presented 
on the conveyor. They are picked up by the robot, and the pickup position is determined by 
CCD. There is no discussion of whether or how the pickup orientation is determined. Kelley 
discusses determining a location of a pickup site on a workpiece. 

Kelley does not discuss selecting one of the images of the reference models that 
matches an image of a plurality of objects, one of which is the object of detection, in the 
fashion discussed above. In contrast, Kelley discusses determining orientation of a 
workpiece held by a vacuum cap hand (28). Column 13, lines 35-53 of Kelley discuss 
"[(following successful acquisition of a workpiece from the hand, workpiece orientation in the 
robot hand is found... If the piece was not dropped, a binary image is formed using the 
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workstation camera (44) and the image features are computed. The features are matched to 
the feature models stored during instruction for each of the J hand-workpiece relationships." 
This process is performed to determine an orientation of the workpiece that is already held by 
the vacuum hand (28), and does not select or find an image of the workpiece that matches 
one of the reference models in a plurality of workpiece is randomly arranged in a container. 
In Kelley, solid regions (corresponding to workpiece services without holes or grooves) in the 
binary image obtained by the arm camera (42) and fun are subjected as candidate hold sites 
for holding a workpiece when being picked up from the been. 

It is particularly notable that Kelley uses a vacuum cap hand. This is because the 
orientation of the workpiece before pickup is not known. The vacuum cap allows pickup 
regardless of the orientation. 

Finally, the rejection's proposed motive to combine Kelley with Suzuki is incorrect. 
The rejection states that it would have been obvious to add Kelley's reference model 
matching in order to acquire a workpiece from randomly arranged workpieces without 
requiring complete knowledge of the position and orientation of a workpiece in a bin. 
However, in Suzuki, the workpieces are laid flat and arrive one at a time on a conveyor belt in 
the order needed for construction. Therefore, there is no need to apply Kelley's technique in 
Suzuki. 



DEPENDENT CLAIMS 

The dependent claims are deemed patentable due at least to their dependence from 
allowable independent claims. These claims are also patentable due to their recitation of 
independently distinguishing features. For example, claims 6, 7, 17, and 18 recites using a 
second image capture device from the orientation found using the reference models to 
further orient the robot. This feature is not taught or suggested by the prior art. Withdrawal 
of the rejection of the dependent claims is respectfully requested. 
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CONCLUSION 

There being no further outstanding objections or rejections, it is submitted that the 
application is in condition for allowance. An early action to that effect is courteously solicited. 

Finally, if there are any formal matters remaining after this response, the Examiner is 
requested to telephone the undersigned to attend to these matters. 

If there are any additional fees associated with filing of this Amendment, please 
charge the same to our Deposit Account No. 19-3935. 



1201 New York Avenue, N.W. 
Suite 700 

Washington, D.C. 20001 
(202)434-1500 
Facsimile: (202)434-1501 



Respectfully submitted, 
STAAS & HALSEY LLP 
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